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METHOD OF DETERMINING MEASUREMENT UNCERTAINTIES 
USING CIRCUIT SIMULATION 

5 CROSS-REFERENCE TO RELATED APPLICATIONS 

[0001] Not applicable. 

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR 

DEVELOPMENT 

10 [0002] Not applicable. 

REFERENCE TO MICROFICHE APPENDIX 
[0003] Not applicable. 

1 5 BACKGROUND OF THE INVENTION 

[0004] Many test systems require periodic calibration to insure that the measurements 
made by the test system are accurate. Although system calibration can significantly 
reduce systemic or "bias" errors, other uncertainties arise that can not be calibrated out, 
and arise in a somewhat random fashion when testing electronic devices. In high- 

20 frequency test systems, measurement uncertainties often arise from impedance 

mismatches between components of the test system and from other sources, such as 
temperature stability, aging, noise, and accuracy and repeatability of the test instruments in 
the test system. Uncertainties from multiple sources can add to or subtract from each 
other to create a total measurement uncertainty for a measurement made by the test 

25 system. 

[0005] In many cases, the measurement uncertainty of a test system or instrument is 
required to be reported along with the measured calibration and/or test values. Standards 
have been developed for calculating and reporting measurement uncertainties. The 
International Organization for Standards ("ISO") provides specifications (e.g. ISO 17025) 

30 for making measurements and reporting uncertainties in a particular fashion. Therefore, 
knowing the uncertainty of a measured value is very important, and sometimes required. 
[0006] Calibration of high-frequency electronic test equipment and test systems is often 
done automatically or semi-automatically. Computer software controls the electronic test 
equipment and can include mathematical equations that calculate the associated 

35 measurement uncertainty ("uncertainty calculations") to provide the test result and 
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associated measurement uncertainty in the desired format. These uncertainty calculations 
are conventionally derived by a highly trained metrologist who examines the specific test 
system being used and develops an uncertainty calculation based on the instrumentation 
and topography of the test system. This process can take several days. 
5 [0007] If a test instrument in the test system is replaced, the values of uncertainties may 
change, but the form of the uncertainty calculations often do not. However, if the 
topography of the test system is changed, such as by adding or removing a switch in the 
signal path, then the form of the uncertainty equations changes and the metrological 
analysis must be re-validated by the metrologist, and the uncertainty calculation 
10 incorporated into the computer test software also must be changed. 

[0008] Therefore, it is desirable to calculate measurement uncertainties for high- 
frequency test systems and instrumentation without manual metrological calculations. 

BRIEF SUMMARY OF THE INVENTION 
15 [0009] A model of a test system including uncertainty terms is developed and entered 
into a simulator. A statistically significant number of iterations of the test system model 
are run for a selected parameter (test value) while varying uncertainty terms within a 
probability distribution, which produces a distribution of test values. The measurement 
uncertainty for a selected parameter is calculated from the distribution of test values. 

20 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0010] Fig. 1 shows a simplified diagram of an exemplary electronic test system. 
[0011] Fig. 2 shows an exemplary screen shot of a circuit simulator operated in 
accordance with an embodiment of the present invention. 
25 [0012] Fig. 3 is a histogram illustrating the results returned from operating the circuit 
simulator of Fig. 2. 

[0013] Fig. 4 is a simplified flow chart of a method of determining measurement 
uncertainties according to an embodiment of the present invention. 

30 DETAILED DESCRIPTION OF THE EMBODIMENTS 

I. Introduction 

[0014] Uncertainties associated with measurements made by electronic test systems are 
determined by simulating the operation of the test system on a simulator. In some 
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embodiments, Monte Carlo analysis is used with a model of the test system run on a 
circuit simulator. The test system model is run many times, varying the values of elements 
in the model that affect measurement uncertainty, to generate a statistical distribution of 
test results. The measurement uncertainties are derived from the statistical distribution of 
5 the test results. 

[0015] Circuit simulators are easily reconfigured to describe the performance of a test 
system topology, and changes to test equipment configurations can be handled in software. 
Similarly, once models for each instrument in a test system have been developed, the 
circuit simulator is capable of calculating measurement uncertainties without the need for 
10 manual derivation of mathematical equations. This means that measurement uncertainties 
can be handled almost entirely by software, without the need for highly trained 
metrologists to develop uncertainty equations. A test technician typically only needs to 
make minor changes to the test system model and re-run the circuit simulation analysis, 
which typically only takes a few seconds for a test system of normal complexity. 

15 

II. An Exemplary Test System 

[0016] Fig. 1 shows a simplified diagram of an exemplary electronic test system 10. 
An input 1 1 of a device under test ("DUT") 12, such as an amplifier, is connected to a 
source 14, such as an RF synthesizer, through cables 16, 18, 20, 22 and switches SI, S2. 

20 An output 13 of the DUT 12 is connected to a receiver 24, such as a spectrum analyzer, 
through cables 26, 28, 30, 32, 34 and switches S3, S4, S5. The switches allow switching 
the output 13 to other loads, such as a power meter head 36, the source 14 to the output 13, 
and generally allows the test system to be configured to perform a variety of 
measurements. A variety of DUTs, such as an attenuator, are alternatively tested, and 

25 alternative test systems are used to test even more types of DUTs, including mixers and 
even test instruments, such as spectrum analyzers. 

[0017] The test system 10 includes a test rack 38 with the source 14, receiver 24, power 
meter 36 ' , and a controller (computer) 40 that controls the other instruments over a bus 
42, and also controls the switches in the test fixture 44. Alternative test systems have 
30 other instruments. Test systems and test fixtures can be much more complex, or more 
simple, than the illustrated example. 

[0018] Deriving the measurement uncertainty calculations (equations) using 
conventional, manual methods is a tedious. The calculations use specified parameters of 
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the components in test system, each of which typically has an associated uncertainty, to 
arrive at a measurement uncertainty. In an embodiment of the present invention, a model 
of the test system running on a circuit simulator establishes measurement uncertainty by 
varying the parameters of the components in the test system within their ranges of 
5 uncertainty. In some embodiments, the parameters are randomly varied within a 
probability distribution. 

III. An Exemplary Simulation 

[0019] Fig. 2 shows an exemplary screen shot 50 of a circuit simulator operated in 
1 0 accordance with an embodiment of the present invention. A simplified test system model 
52 includes a numerical representation (model) of the source 14, DUT 12, and other 
components of a test system. The source 14 is connected to the DUT 12 with a 
transmission line 54 and the DUT 12 is connected to the receiver 24 with a transmission 
line 56. Alternatively, a test system model includes individual switches and cable 
15 segments, with their associated uncertainties, instead of simple transmission lines 54, 56. 
The topology of the test system model of Fig. 2 was created manually, but circuit 
simulation technology easily accepts automatically generated circuit/system specifications, 
enabling full automation of the modeling process. 

[0020] Conventional manual techniques often ignore the contributions to measurement 
20 uncertainty made by components such as cables and switches for two reasons. First, the 
components typically have impedance close to fifty ohms, and thus contribute little to the 
overall uncertainty in some test systems. Second, including each component in a 
conventional uncertainty calculation quickly becomes unwieldy. However, a more 
accurate characterization of measurement uncertainty is obtained if all components of the 
25 system are included in a model of the test system for simulation. More accurate 

characterizations of measurement uncertainty are particularly desirable as operating 
parameters change, such as with an increase in frequency or a decrease in power, and/or 
for devices with stringent performance requirements {i.e. when typical performance limits 
are close to the specification limits). 
30 [0021] The screen shot 50 is taken from an ADS™ ("Advanced Design System") 

circuit simulator available from AGILENT TECHNOLOGIES, INC., of Palo Alto, California, 
but several commercially available circuit simulators (generally known as "simulators"), 
such as SPICE simulators, are suitable for use in alternative embodiments of the invention. 
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The ADS™ simulator used a harmonic balance analysis to perform the simulations in the 
frequency domain, but an ADS™ system can perform other types of analyses, such as 
analyses in the time domain, linear S-parameter analysis, Ptolemy analysis, and 
electromagnetic analysis. Linear S-parameter analysis is desirable when testing relatively 
5 simple, linear devices, such as attenuators, where most of the uncertainty arises from 
impedance mismatch, because S-parameter analysis is quick and efficient for such a 
simulation. 

[0022] The screen shot shows the parameters entered into the test system model 52 for 
the various components, and the simulator settings 58. For example, the signal source 14 

10 is modeled as providing a 1 GHz single tone of IV (V=l) with a statistical Gaussian 

variation of ±1%. A Gaussian distribution is generally the default choice for performance 
variables because a component is probably closer to the nominal value than to the edge of 
a specification window. However, simulators can handle a variety of non-Gaussian 
distributions for performance variables. The first transmission line 54 has a nominal 

1 5 impedance of fifty ohms (Z=50) with a ±1% Gaussian variation, and an electrical length of 
ninety degrees (E=90). The test system model 52 is shown in a graphical user interface. 
Test system models used in alternative embodiments are expressed as a net list. 
[0023] The transmission line and other test system components are modeled at 1 GHz, 
but could be modeled at other frequencies. The performance values and ranges of 

20 components in the test system model typically change with frequency, power, and other 
factors, such as system temperature. Modeling a test system on a circuit simulator allows 
one to easily determine measurement uncertainty under several operating conditions, thus 
taking into account frequency- power and/or temperature-dependent contributions to the 
overall uncertainty from test system components. Those of skill in the art will appreciate 

25 that other factors affecting measurement uncertainty may be present, and that frequency, 
power, and operating temperature are merely exemplary. 

[0024] For example, the power measurement accuracy of a receiver might be greater at 
one input power than another, or the accuracy and/or repeatability of a source might be 
better at one frequency than another. Performing the simulations different operating 
30 conditions with the appropriate values entered into the model provides different 
uncertainty distributions for the same topological model. 
[0025] The DUT 12 is modeled as an amplifier with ten dB of gain 
(S2i=dbpolar(10,0)), perfect input match (Sn=polar(0,0)), perfect output match 
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(Sii^pola^OJSO)), and infinite reverse isolation (Si2=0). When modeling an actual test 
system and DUT, the s-parameters, and the ranges and distributions of the s-parameters, 
would typically be non-zero values. This exemplary model is highly simplified for 
purposes of illustration. 
5 [0026] Similarly, the second transmission line 56 is modeled as having an impedance 
of fifty ohms (with no variability), and an electrical length of ninety degrees (with no 
variability), i.e.,, the second transmission line is modeled as a perfect transmission line for 
simplicity of illustration. The measurement receiver 24 includes a noise source 60 and a 
fifty-ohm termination (Rl) 62. The noise component is modeled as a signal source 

10 providing a second 1 GHz signal that varies ±0.01 V around 0.0 volts in a Gaussian 

fashion. Thus, the test system model 52 can handle many types of sources of potential 
uncertainty, such as impedance mismatch, variations in electrical length, generated noise, 
temperature and/or aging drift of the test instruments, and known (either specified or 
measured) repeatability and accuracy of the test instruments. Simulators can also handle 

15 non-linearities, and easily handle complex topologies, including test system components in 
the simulator model that might be ignored in a manual uncertainty calculation. The values 
chosen in the test system model are merely for purposes of illustration. 
[0027] In particular, the mismatches of switches, cables, connector ends, and similar 
low-variation components tend to be ignored when manually deriving uncertainty 

20 calculations. However, such mismatches can become important when testing DUTs at 
very high and ultra high frequencies. For example, an uncertainty calculation for an 
integrated circuit ("IC") operating at 300 MHz might need only a relatively simple 
uncertainty calculation, but a similar IC operating at several GHz might need a much more 
complex (complete) uncertainty calculation to accurately characterize the test system. 

25 [0028] The simulator settings 58 are set so that 1000 iterations of the test system model 
52 are run using Monte Carlo techniques to vary the four variable parameters (i.e. 
uncertainty terms). Generally, a Monte Carlo simulation selects variable values at random 
to simulate a model, named after the games of chance at the Monte Carlo casino. When 
you roll a die, you know that either 1 , 2, 3, 4, 5, or 6 will come up, but you don't know 

30 which value will come up for any particular roll. It is similar with the uncertainty terms, 
which have a known range of values but an uncertain value for any event {e.g. iteration of 
the test system model). Monte Carlo techniques randomly generate values for variables 
within a probability distribution, examples of which include Gaussian, rectangular, 
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triangular, and lognormal, and run a model (solve a problem) over and over to provide a 
distribution of solutions {e.g. test results). Many simulators include the ability to perform 
Monte Carlo analyses. Simulators can be configured to provide many different types of 
information, such as yield to a specification. In this case, the gain of the amplifier (DUT 
5 12) is the parameter of interest, so the YIELD and YIELD SPEC are set so that all results 
(solutions) from the 1000 iterations are saved (Le. all iterations "pass"). More information 
on entering circuit models and setting simulator parameters is found in the ADS 
Characterization and Modeling Handbook, available from Agilent Technologies, Inc. 
[0029] A harmonic balance analysis is done for the first order (i.e. fundamental) 

10 frequency, which is 1 GHz. Uncertainty analysis could also be done for the harmonic 
frequencies, and in the case where the DUT is a mixer, on the mixing products and 
harmonics of the mixing products. Monte Carlo analyses are independent of the 
underlying circuit simulation technology, so embodiments of the invention can be used 
with other types of simulation engines, such as time-domain, circuit envelope, or system- 

1 5 level simulators, for example. In further embodiments, different types of measurements 
on the same DUT use different types of simulation engines, the types of simulation 
engines being chosen to best match the type of measurement being evaluated. 
[0030] With the four variable parameters (V of the signal source 14, Z of the first 
transmission line 54, E of the first transmission line 54, and V of the noise source 60), the 

20 simulator ran 1000 iterations in approximately one second on an 800 MHz personal 

computer. The simulator can display the results in a variety of formats, and can display 
any of a variety of selected parameters. The voltage output of the amplifier is the 
parameter used in this example to illustrate determining measurement uncertainty. 
[0031] Fig. 3 is a histogram 80 illustrating the results returned from operating the 

25 circuit simulator of Fig. 2. A histogram is one of several ways to display the probability 
distribution of the solutions (results) of the iterations. A ten dB voltage gain corresponds 
to a twenty-times increase in voltage. In other words, ten dB of gain increases a 1 V 
signal to a 20 V signal. Thus, the test variable (output voltage from the amplifier on a 
linear scale) is centered around 20 volts. The histogram shows how many solutions fall 

30 within each range. For example, about 245 of the 1000 iterations resulted in an output 
voltage solution within range 80. About 50 of the 1000 iterations resulted in an output 
voltage solution within range 84. 

[0032] Many statistical values may be calculated from the solutions generated by the 
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1 000 iterations of the simulator. For example, the mean and standard deviation of the 
1000 solutions are calculated. Standard deviations provide an indication of what 
percentage of occurrences will fall within a window. If the quantity in question is 
modeled by a normal probability distribution, there are no finite limits that will contain 
5 100 % of its possible values. However, plus and minus 3 standard deviations about the 
mean of a normal distribution corresponds to 99.73 % limits. Measurement uncertainties 
are easily calculated from the probability distribution generated by the simulator. Various 
limits could be determined, such as what limits would include 95% of the solutions. In 
other words, provide a 95% confidence level (certainty) that the selected parameter (e.g. 

10 gain) would fall within the limits. In some instances, such as where greater precision or 
confidence is required, more iterations are performed. Generally, enough iterations are 
run to produce a statistically significant sample. The number of iterations that provide 
statistical significance vary depending on many factors, and can be empirically 
determined, if necessary, by running different numbers of iterations and comparing the 

1 5 statistical values, for example. 

IV. Exemplary Methods 

[0033] Fig. 4 is a simplified flow chart of a method of determining measurement 
uncertainties according to an embodiment of the present invention. A model of a test 

20 system ("test system model") including a plurality of uncertainty terms is developed (step 
402). Optionally, a computer-readable library of test system components, such as 
instruments, cables, and switches, with their associated uncertainty terms, is developed 
(step 404). The test system model is entered into a simulator (step 406). In a particular 
embodiment, entering the test system model into the simulator includes automatically 

25 loading uncertainty terms associated with test system components from the computer- 
readable library. 

[0034] A sufficient number of iterations are run on the simulator varying each of a first 
portion of the plurality of uncertainty terms to produce a statistically significant number of 
results of a selected parameter (step 408). The first portion of the plurality of uncertainty 
30 terms are those uncertainty terms in the test system model that are relevant to the selected 
test result that is being simulated. There is usually no reason to vary the other uncertainty 
terms, such as uncertainty terms for instruments or paths that not used when simulating the 
selected the selected parameter. The results are evaluated to determine a measurement 



Atty. Docket No. 10030906-1 



uncertainty of the selected parameter (step 410). Some embodiments automatically 
calculate the measurement uncertainty from the distribution of results. 
[0035] In some embodiments, the selected parameter and hence measurement 
uncertainty is evaluated at a first operating condition of the test system or DUT, such as at 
5 a particular frequency, gain, DUT bias point, or power into or out of the DUT. In a further 
embodiment, the selected parameter is re-evaluated at a second operating condition 
(branch 412). 

[0036] While the preferred embodiments of the present invention have been illustrated 
in detail, it should be apparent that modifications and adaptations to these embodiments 
10 might occur to one skilled in the art without departing from the scope of the present 
invention as set forth in the following claims. 
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